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1.  INTRODUCTION 


In  industry  today,  there  is  considerable  emphasis  on  attempting  to  quantify,  qualify,  and  define 
systems  engineering  and  the  systems  engineering  process.  Systems  Engineering  Process  Assessments 
(SEP As)  and  the  like,  are  being  performed  to  identify  the  relative  maturity  of  the  process  (e.g.,  how 
well  do  processes  match  up  with  a  macro  model  and  how  well  are  they  institutionalized  and  used). 
While  these  assessments  can  be  powerful  tools,  they  do  not  provide  insight  into  the  methodology  of 
systems  engineering  or  the  effectiveness  of  the  systems  engineering  process.  Only  when  the  maturity 
level,  methods,  tools,  and  effectiveness  are  viewed  in  the  aggregate,  can  the  systems  engineering 
process  be  truly  assessed.  Figure  1  shows  this  aggregate  view  of  systems  engineering. 


Figure  1 .  Overview  of  Systems  Engineering 


1 


Introduction 


One  method  of  obtaining  this  perspective  is  through  benchmarking  the  systems  engineering  process. 
The  following  list  describes  several  key  benefits  of  performing  a  benchmarking  effort  of  this  type: 

•  Identifies  the  state  of  practice  of  systems  engineering: 

-  Determines  levels  of  maturity  of  the  organization 

-  Demonstrates  how  organizations  compare 

-  Provides  a  baseline  to  investigate  alternates  in  systems  engineering 

-  Provides  the  means  to  assess  the  changes  due  to  the  use  of  alternate  systems  engineering 
methodologies,  tools,  and  processes 

•  Identifies  the  present  systems  engineering  methodologies: 

-  Provides  a  basis  to  develop  new  methodologies 

-  Provides  the  means  to  measure  improvement 

•  Identifies  the  predominant  systems  engineering  tools 

•  Correlates  practices,  methodologies,  and  tools  with  process  maturity  level  and  marketplaces 
Other  benefits  of  benchmarking  the  systems  engineering  process  include: 

•  Provides  systems  engineering  process  model  exposure  (e.g.,  for  an  organization  preparing 
for  a  SEPA 

•  Identifies  expertise  in  the  process  areas 

•  Provides  a  tool  for  process  improvement 

•  Provides  a  means  to  compare  groups  and  organizations  against  each  other 

1.1  SCOPE 

This  technical  report  provides  a  brief  overview  of  the  use  and  application  of  the  Systems  Engineering 
Maturity  and  Benchmarking  Questionnaire.  The  questionnaire  is  a  vehicle  used  to  assess  the  maturity 
level  of  the  systems  engineering  process  and  to  identify  the  tools,  practices,  and  methodologies  that 
are  used  in  an  organization. 

1.2  AUDIENCE 

The  intended  audience  for  this  technical  report  includes  those  individuals  involved  with  systems 
engineering  organizational  development,  process  definition,  process  improvement,  and  SEPA 
preparation. 

1.3  ORGANIZATION  OF  THE  TECHNICAL  REPORT 

This  report  is  organized  in  the  following  manner: 

•  Section  2  presents  an  overview  of  the  objectives,  basis,  and  structure  of  the  questionnaire. 
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•  Section  3  describes  the  various  applications  that  the  questionnaire  can  be  used  to  support. 

•  Section  4  presents  an  overview  of  how  to  tailor  and  administer  the  questionnaire  and  how  to 
interpret  the  results. 

•  Appendix  A  presents  an  overview  of  the  available  systems  engineering  process  models. 

•  Appendix  B  presents  the  Systems  Engineering  Maturity  and  Benchmarking  Questionnaire. 

1.4  TYPOGRAPHIC  CONVENTIONS 

This  report  uses  the  following  typographic  conventions: 


Serif  font . General  presentation  of  information. 

Italicized  serif  iotA . Publication  titles. 

Boldfaced  serif  font . Section  headings  and  emphasis. 
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Introduction 


2.  QUESTIONNAIRE  OVERVIEW 

2.1  OBJECTIVES 

The  objectives  of  using  the  Systesm  Engineering  Maturity  and  Benchmarking  Questionnaire  are: 

•  To  gather  a  broad  understanding  of  the  level  of  maturity  of  the  systems  engineering  process 
in  use  today 

•  To  understand  the  tools  and  methods  in  use 

•  To  identify  (if  any)  groupings  (domains) 

•  To  investigate  a  possible  correlation  between  level  of  maturity,  tools  used,  and  methodologies 

2.2  BASIS 

The  basis  of  the  questionnaire  is  the  Systems  Engineering  Capability  Maturity  Model  (SE-CMM™), 
Version  1.1,  which  was  developed  as  a  collaborative  effort  among  industry,  academia,  and 
government  entities.  The  SE-CMM  is  intended  to  support  process  improvement  efforts  through 
appraisal  of  an  organization’s  systems  engineering  process.  The  model  is  based  on  18  Process  Areas 
(PAs)  covering  engineering,  project,  and  organization  categories.  Table  1  shows  the  mapping  between 
the  PAs  and  the  categories.  For  each  PA,  there  are  associated  capability  levels  or  maturity  levels 
identifying  how  well  an  organization  performs  the  practice.  Table  2  shows  the  common  features  for 
each  maturity  level. 
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2.  Questionnaire  Overview 


Table  1.  Process  Areas  by  Category 


PA  01 

Analyze  Candidate  Solutions 

PA  02 

Derive  and  Allocate  Requirements 

PA  03 

Evolve  System  Architecture 

PA  04 

Integrate  Disciplines 

PA  05 

Integrate  System 

PA  06 

Understand  Customer  Needs  and  Expectations 

PA  07 

Verify  and  Validate  System 

Pro 

_ 1 

1 

PA  08 

PA  09 

Mange  Configurations 

PA  10 

Mange  Risk 

PA  11 

Monitor  and  Control  Technical  Effort 

PA  12 

Plan  Technical  Effort 

1  Organization  | 

1 

PA  13 

Define  Organization’s  Systems  Engineering  Process 

PA  14 

Improve  Organization’s  Systems  Engineering  Processes 

PA  15 

Manage  Product  Line  Evolution 

PA  16 

Manage  Systems  Engineering  Support  Environment 

PA  17 

Provide  Ongoing  Skills  and  Knowledge 

PA  18 

Coordinate  With  Suppliers 

Table  2.  Levels  of  Maturity  and  Common  Features 


Maturity 

Level 

Capability  Level 

Common  Features 

0 

Not  Performed 

1 

•  Base  practices  performed 

2 

Planned  and  Tracked 

•  Committing  to  perform 

•  Planning  performance 

•  Disciplined  performance 

•  Tracking  performance 

•  Verify  performance 

3 

Well  Defined 

•  Defining  a  standard  process 

•  Tailoring  the  standard  process 

•  Using  data 

•  Perform  the  defined  process 

4 

Quantitatively 

Controlled 

•  Establishing  measurable  quality  goals 

•  Determining  process  capability  to  achieve  goals 

•  Objectively  managing  performance 

5 

•  Establishing  quantitative  process  effectiveness  goals 

•  Improving  process  effectiveness 

2.3  STRUCTURE 

The  questionnaire  contains  four  sections: 

1.  Organizational  Information.  This  section  contains  general  information  about  the 
respondent  and  the  organization.  This  information  is  used  for  logistical  purposes  only 
and  is  not  typically  contained  in  the  data  analysis  sample. 

2.  Systems  Engineering  Overview.  This  section  attempts  to  capture  a  general  overview  of  the 
systems  engineering  process  within  an  organization. 
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3.  Key  Systems  Engineering  Process  Areas.  This  section  gathers  maturity  data,  tools,  and 
methods  used  for  each  of  the  18  PAs.  For  each  PA,  a  common  template  (as  shown  in 
Table  3)  is  used  to  address  several  questions. 


Table  3.  Common  Questionnaire  Template 


Process 

Adequacy 

•  Is  the  process  area  generally  performed  to  a  sufficient  level  to  ensure  a 
cost-effective  system  that  satisfies  the  customer’s  requirements? 

•  What  are  the  limiting  factors  for  the  process  area? 

Applied 

Methodology 

•  What  methods  are  generally  performed  for  the  process  area? 

•  What  tools  are  generally  used? 

•  What  internal,  industry,  and  government  standards  are  used? 

•  What  metrics  are  used  to  monitor  the  process? 

Process 

Improvement 

•  What  are  the  two  limiting  items  for  the  process? 

4.  Survey  Feedback  Form.  This  section  allows  the  respondents  to  provide  feedback  on  the 
questionnaire  for  future  updates. 


2.  Questionnaire  Overview 


This  page  intentionally  left  blank. 
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3.  USING  THE  QUESTIONNAIRE 


3.1  ASSESSMENT  PREPARATION 

As  more  organizations  are  becoming  involved  in  the  assessment  of  their  systems  engineering  process, 
there  is  a  growing  desire  for  a  low  cost  way  to  prepare  for  it.  Because  the  Systems  Engineering 
Maturity  and  Benchmarking  is  based  on  the  SE-CMM,  the  questionnaire  provides  the  means  to 
acquaint  the  organization  with  the  SEPA  process.  Analysis  of  the  results  of  the  questionnaire  can 
provide  insight  into  areas  where  additional  training  is  required,  identify  process  weaknesses  and 
strengths  of  the  organization,  and  provide  an  overview  of  the  relative  maturity  of  the  organization. 


3.2  BENCHMARKING 

The  primary  objective  of  the  questionnaire  is  to  provide  a  tool  to  benchmark  key  information 
concerning  processes,  methodologies,  tools,  and  metrics  in  a  common  framework.  Depending  on  the 
needs  of  the  survey  team,  the  questionnaire  can  be  used  to  assess  the  systems  engineering  process  at 
various  levels  of  abstraction  as  shown  in  Table  4. 


Table  4.  Information  Gathered  at  Various  Levels  of  Abstraction 


Level  of  Abstraction 

Information  Gathered 

Organizational 

•  Compare  various  groups/programs 

•  Baseline  the  present  system  engineering  process 

•  Develop  an  understanding  of  the  state  of  practices  within  the 
organization 

Company 

•  Compare  organizations  within  a  company 

•  Identify  areas  of  process  expertise 

•  Develop  an  understanding  of  the  state  of  practices  within  the  company 

Corporate 

•  Compare  companies  within  a  corporation 

•  Identify  areas  of  process  expertise 

•  Develop  an  understanding  of  the  state  of  practices  within  the 
corporation 

Industry 

•  Compare  corporations  and  industries 

•  Identify  areas  of  process  expertise 

•  Develop  an  understanding  of  the  state  of  practices  of  systems 
engineering  in  the  industry  at  large 
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3.  Using  the  Questionnaire 


3.3  PROCESS  IMPROVEMENT  THROUGH  BENCHMARKING 

The  classic  definition  of  benchmarking  is  to  compare  something  to  a  standard.  While  this  definition  is 
correct,  its  static  nature  limits  benchmarking  to  that  of  a  measure  as  opposed  to  a  driver  of  change. 
The  definition  of  benchmarking  in  the  active  form  is: 

A  continuous  process  of  measuring  products,  services  and  practices  against  the  toughest  competitors  or 
those . . .  recognized  as  leaders. 

David  T.  Kearns,  CEO,  Xerox  Corporation 


In  other  words,  benchmarking  is  a  continuing  process  of: 

•  Identifying  leading  organizations  (projects,  companies,  or  industries)  with  complementary 
products,  processes,  and  methodologies 

•  Comparing  the  current  organization  against  these  leading  organizations 

•  Learning  were  similarities  and  differences  exist  among  the  compared  organizations 

•  Understanding  the  differences  among  the  compared  organizations 

•  Evolving  an  organization’s  present  process  with  what  was  learned 

•  Implementing  the  new  process 

•  Refining  the  process  to  develop  organizational  best  practices 
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4.  SURVEY 


4.1  TAILORING/ADMINISTRATION/DATA  REDUCTION 

The  questionnaire  was  developed  to  be  tailorable  and  easily  administered  by  a  survey  team.  Figure  2 
demonstrates  the  overall  tailoring/administration/data  reduction  effort. 

The  survey  process  can  be  seen  as  a  two  step  process.  The  first  is  to  survey  a  small  group  of 
respondents  with  an  exploratory  questionnaire.  This  allows  the  survey  team  to  identify  groupings  or 
domains  in  the  response.  These  groupings  are  used  to  replace  the  exploratory  (textual)  responses  with 
quantifiable  selections  (i.e.,  check  a  box).  The  updated  questionnaire  is  then  given  out  to  a  broad 
group,  and  the  results  are  reduced  using  standard  tools  and  techniques. 

4.1.1  Tailoring 

As  shown  in  Figure  2,  there  are  four  primary  areas  (T1  through  T4)  to  tailor  the  questionnaire.  They 
are: 


T1  Replacing  the  general  terminology  and  references  to  project/organization/company  with 
specific  ones 

This  tailoring  step  can  provide  significant  benefit  in  the  survey  process  by  reducing  the 
need  for  clarification  by  the  survey  team  and  reducing  the  uncertainty  on  the  part  of 
the  respondent.  The  net  result  is  a  questionnaire  that  is  quicker  to  fill  out  and  easier  to 
understand  and  produces  higher  quality  results  by  reducing  confusion. 

T2  Down  selecting  PAs  based  on  organizational  goals 

Depending  on  the  organization  and  the  survey  goals,  only  a  subset  of  process  areas  may 
be  addressed.  This  down  selection  can  be  based  on  the  fact  that  certain  organizations 
may  perform  only  select  PAs,  or  the  organization  may  want  to  concentrate  in  select 
PAs. 

T3  Replacing  exploratory  (textual)  responses  with  quantifiable  selections 

There  is  an  initial  requirement  to  use  a  textual  based  questionnaire  to  identify 
groupings  or  domains  of  tools,  methods,  and  metrics.  Once  these  have  been  identified, 
the  textual  response  can  be  replaced  by  a  delineated  list  of  “common”  elements.  The 
questionnaire  may  be  directly  updated,  or  a  “tickle  list”  may  be  provided  separately.  It 
is  important  to  note  that  due  to  the  rapid  changes  in  the  marketplace  these  “common” 
elements  age  rapidly. 


II 


4.  Survey 


Depending  on  the  number  of  planned  respondents,  physical  location,  survey  phase,  tools  available, 
and  the  organizational  goals,  the  questionnaire  can  be  administered  in  several  different  forms.  The 
following  survey  administration  methods  may  be  used: 

Hard  copy  Photocopies  of  the  questionnaire.  Hard  copy  is  the  simplest  method  to 
administer  (hand  or  mail  out,  no  special  tools  are  required)  and  the  most 
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4.  Survey 


difficult  to  fill  out  and  perform  data  reduction  on.  The  use  of  hard  copy 
should  be  limited  as  much  as  possible. 

Online  Interactive  questionnaire  available  via  local  area  networks  and  wide  area 

networks  (e.g.,  Internet).  Assuming  the  respondent  has  access,  online  is  the 
simplest  method  for  both  the  respondent  and  the  survey  team.  Collected 
data  is  automatically  placed  into  a  centralized  data  repository  for  data 
reduction  and  analysis. 

One-on-one  One-on-one  between  a  survey  team  member  and  the  respondent. 

One-on-one  is  the  most  time  consuming  method  available,  but  it  gives  the 
highest  quality  results.  This  method  is  very  useful  during  the  initial  survey 
to  not  only  identify  groupings  and  domains,  but  to  understand  where  there 
is  a  shortcoming  in  the  questionnaire.  During  the  broad  survey,  one-on-one 
may  be  used  as  a  follow-up  method  to  gather  detailed  information  in  the 
support  of  process  improvement  and  the  identification  of  centers  of 
excellence. 

Phone  Similar  to  the  one-on-one,  but  done  over  the  phone.  Although  not  as 

effective  as  the  one-on-one  method,  phone  surveys  provide  good  results  at 
lower  cost  by  eliminating  the  need  for  travel. 

Soft  copy  Copy  of  the  questionnaire  on  diskette.  Similar  to  the  online  method,  but 

electronic  copies  of  the  questionnaire  are  handed  or  mailed  out  to  the 
respondents.  The  main  drawback  with  this  method  is  the  time  delay  involved 
with  mailing  to  a  location,  mailing  it  back,  and  then  uploading  the  data  into 
a  central  repository  for  data  reduction  and  analysis.  While  not  as  automated 
as  the  online  method,  it  eliminates  the  need  for  network  access,  development 
of  an  online  questionnaire,  and  concerns  of  data  access  on  a  nonsecured 
network. 

4.1.3  Data  Reduction 

During  the  initial  small  group  survey,  much  of  the  data  collected  is  in  the  form  of  textual  responses. 
While  textual  responses  contain  a  high  level  of  information,  they  do  not  lend  themselves  to  traditional 
data  analysis  techniques.  The  survey  team  needs  to  interpret  the  textual  responses  in  an  attempt  to 
identify  groupings  or  domains.  Depending  on  the  expertise  of  the  survey  team,  discipline  experts 
may  be  called  on  to  support  this  interpretation.  When  the  groupings  or  domains  are  identified,  they 
need  to  be  fed  back  to  the  respondents  to  verify  that  the  groupings  or  domains  adequately  capture 
the  intent. 

During  the  broad  survey,  the  amount  of  textual  responses  should  be  minimal  to  allow  for  more 
traditional  data  reduction  and  analysis  techniques.  Some  typical  analyses  that  would  be  performed 
include: 


•  Distribution  of  maturity  levels 

•  Distribution  of  drivers 
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4.  Survey 


•  Distribution  of  tools,  methods,  and  metrics 

•  Correlation  between  the  various  distributions 

•  Identification  of  centers  of  excellence 

Additional  analyses  may  be  performed  to  compare  previous  survey  results  with  the  present  results. 
The  objective  of  this  data  analysis  is  to  support  the  process  improvement  effort  by  identifying 
developments  in  tools,  methods,  metrics,  drivers,  and  maturity  level  and  associated  trends. 

4.2  SECURITY 

For  most  organizations,  the  questionnaire  data  needs  to  be  maintained  in  an  anonymous  fashion.  This 
level  of  confidentiality  provides  a  protective  blanket  for  the  respondent  to  alleviate  any  apprehension 
in  recording  “what  really  goes  on”  verses  “what  is  politically  correct.”  It  is  up  to  the  survey  team  to 
foster  this  open  and  frank  environment  with  the  respondents.  The  survey  team  can  create  an  open 
atmosphere  by  being  up  front  with  the  respondents  concerning  the  objective  of  the  survey  and  what 
will  happen.  The  team  can  also  assure  respondents  that: 

•  No  project  or  individual  will  be  identified  in  the  findings. 

•  No  discussion  of  the  comments  will  occur  outside  the  survey. 

•  All  raw  data  will  be  destroyed  after  the  survey  is  performed  (typically). 

In  some  cases,  a  member  of  the  survey  team  wUl  have  a  “key”  that  maps  respondents  to  the 
individuals.  This  key  is  used  when  specific  data  needs  to  be  gathered.  For  example,  when  locating 
process  expertise,  the  survey  team  would  want  to  go  back  to  the  process  expert  to  document  and 
better  understand  the  process. 
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APPENDIX  A.  SYSTEMS  ENGINEERING 
PROCESS  MODELS 


A.1  PROCESS  MODEL  DESCRIPTIONS 

At  present,  there  are  two  common  systems  engineering  process  models:  SEI’s  SE-CMM  and 
INCOSE’s  Systems  Engineering  Cabability  Assessment  Model  (SECAM).  The  basic  structure  of 
SE-CMM  and  the  SECAM  are  similar,  with  each  assessing  the  systems  engineering  process  through  a 
series  of  PAs  (SE-CMM)  or  Key  Focus  Areas  (KFAs)  (SECAM).  Tables  5  and  6  provide  a  mapping 
between  PAs  and  KFAs. 


Table  5.  Mapping  of  SE-CMM  Process  Areas  and  SECAM  Key  Focus  Areas 


SE-CMM  (Vl.l) 

Process  Areas 

SECAM  (Vl.O) 

Key  Focus  Areas 

PA  01 

Analyze  Candidate  Solutions 

KFA  3.4 

Integrated  Engineering  Analysis 

PA  02 

Derive  and  Allocate  Requirements 

KFA  3.2 

System  Requirements 

PA  03 

Evolve  System  Architecture 

KFA  3.3 

System  Design 

PA  04 

Integrate  Disciplines 

KFA  1.4 

Intergroup  Coordination 

PA  05 

Integrate  System 

KFA  3.5 

System  Integration 

PA  06 

Understand  Customer  Needs  and 
Expectations 

KFA  3.1 

System  Concept  Definition 

PA  07 

Verify  and  Validate  System 

KFA  3.6 
KFA  3.7 

System  Verification 

System  Validation 

KFA  1.6 

Quality  Assurance 

PA  09 

Manage  Configurations 

KFA  1.5 

Configuration  Management 

PA  10 

Manage  Risk 

KFA  1.7 

Risk  Management 

PA  11 

Monitor  and  Control  Technical  Effort 

KFA  1.2 

Tracking  and  Oversight 

PA  12 

Plan  Technical  Effort 

KFA  1.1 

Planning 

PA  13 

Define  Organization’s  Systems 
Engineering  Process 

KFA  2.1 

Process  Management  and  Improvement 

PA  14 

Improve  Organization’s  Systems 
Engineering  Processes 

PA  15 

Manage  Product  Line  Evolution 

Technology  Management 

PA  16 

Manage  Systems  Engineering  Support 
Environment 

KFA  2.4 

Environment  and  Tool  Support 

PA  17 

Provide  Ongoing  Skills  and  Knowledge 

KFA  2.2 

Training 

PA  18 

Coordinate  With  Suppliers 

KFA  1.3 

Subcontract  Management 
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Table  6.  Mapping  of  SECAM  Key  Focus  Areas  and  SE-CMM  Process  Areas 


SECAM  (Vl.O) 

Key  Focus  Areas 

SE-CMM  (Vl.l) 

Process  Areas 

KFA  1.1 

Planning 

PA  12 

Plan  Technical  Effort 

KFA  1.2 

Tracking  and  Oversight 

PA  11 

Monitor  and  Control  Technical  Effort 

KFA  1.3 

Subcontract  Management 

PA  18 

Coordinate  With  Suppliers 

KFA  1.4 

Intergroup  Coordination 

PA  04 

Integrate  Disciplines 

KFA  1.5 

Configuration  Management 

PA  09 

Manage  Configurations 

KFA  1.6 

Quality  Assurance 

KFA  1.7 

Risk  Management 

PA  10 

Manage  Risk 

KFA  2.1 

Process  Management  and 

Improvement 

PA  13 

Define  Organization’s  Systems 
Engineering  Process 

PA  14 

Improve  Organization’s  Systems 
Engineering  Processes 

KFA  2.2 

Training 

PA  17 

Provide  Ongoing  Skills  and  Knowledge 

KFA  2.3 

Technology  Management 

PA  15 

Manage  Product  Line  Evolution 

KFA  2.4 

Environment  and  Tool  Support 

PA  16 

Manage  Systems  Engineering  Support 
Environment 

KFA  3.1 

System  Concept  Definition 

PA  06 

Understand  Customer  Needs  and 
Expectations 

KFA  3.2 

System  Requirements 

PA  02 

Derive  and  Allocate  Requirements 

KFA  3.3 

System  Design 

PA  03 

Evolve  System  Architecture 

KFA  3.4 

Integrated  Engineering  Analysis 

PA  01 

Analyze  Candidate  Solutions 

KFA  3.5 

System  Integration 

PA  05 

Integrate  System 

KFA  3.6 

System  Verification 

PA  07 

Verify  and  Validate  System 

KFA  3.7 

System  Validation 

PA  07 

Verify  and  Validate  System 

A.2  ORDERING  INFORMATION 

The  SE-CMM  (Systems  Engineering  Capability 
available  from  the  following  sources: 

Software  Productivity  Consortium 
SPC  Building 
2214  Rock  Hill  Road 
Herndon,  Virginia  22070 

1-800-827-4SPC 

Electronic  copy  is  also  available 
online.  Please  contact  the  Consortium 
for  more  information. 

Defense  Technical  Information  Center 
(DTIC) 

Attn:  DTIC-OCP 

8725  John  J.  Kingman  Road 

Suite  0944 

Ft.  Belvoir,  Virginia  22060-6218 

703-767-8019/8021/8022/8023 

Research  Access,  Inc. 

800  Vinial  Street 
Pittsburgh,  Pennsylvania  15212 

1-800-685-6510 


Maturity  Model,  VI. 1,  CMU/SEI-95-MM-003)  is 


Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  Pennsylvania  15213 

412-268-2000 

Electronic  copy  is  also  available 
online.  Please  contact  SEI  for  more 
information. 


National  Technical  Information  Service 
(NTIS) 

U.S.  Department  of  Commerce 
Springfield,  Virginia  22161 

703-487-4600 


16 


Appendix  A.  Systems  Engineering  Process  Models 


The  SECAM  (Systems  Engineering  Capability  Assessment  Model)  is  available  from  the  following 
source: 

International  Council  on  Systems  Engineering  (INCOSE) 

2033  Sixth  Avenue,  Suite  804 
Seattle,  Washington  98121 


206-441-1164  or  800-366-1164 


Appendix  A.  Systems  Engineering  Process  Models 


This  page  intentionally  left  blank. 
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LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


INCOSE 

International  Council  on  Systems  Engineering 

KFA 

Key  Focus  Area 

PA 

Process  Area 

SECAM 

System  Engineering  Capability  Assessment  Model 

SE-CMM 

Systems  Engineering  Capability  Maturity  Model 

SEI 

Software  Engineering  Institute 

SEPA 

Systems  Engineering  Process  Assessment 
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APPENDIX  B.  SYSTEMS  ENGINEERING 
MATURITY  AND  BENCHMARKING 
QUESTIONNAIRE 


Produced  by  the 

SOFTWARE  PRODUCTIVITY  CONSORTIUM 
SPC  Building 
2214  Rock  Hill  Road 
Herndon,  Virginia  22070 


Copyright  ©  1996,  Software  Productivity  Consortium  NFP,  Inc.,  Herndon,  Virginia.  All  rights  reserved.  Permission  to  use, 
copy,  modify,  and  distribute  this  material  for  any  purpose  and  without  fee  is  hereby  granted  consistent  with  48  CFR  227 
and  252,  and  provided  that  the  above  copyright  notice  appears  in  all  copies  and  that  both  this  copyright  notice  and  this 
permission  notice  appear  in  supporting  documentation.  This  material  is  based  in  part  upon  work  sponsored  by  the  Defense 
Advanced  Research  Projects  Agency  under  Grant  #MDA972“92“J-1018.  The  content  does  not  necessarily  reflect  the 
position  or  the  policy  of  the  U.S.  Government,  and  no  official  endorsement  should  be  inferred.  The  name  Software 
Productivity  Consortium  NFP,  Inc.  shall  not  be  used  in  advertising  or  publicity  pertaining  to  this  material  or  otherwise 
without  the  prior  written  permission  of  Software  Productivity  Consortium  WP,  Inc.  SOFTWARE  PRODUCTIVITY 
CONSORTIUM  NFP,  INC.  MAKE  NO  REPRESENTATIONS  OR  WARRANTIES  ABOUT  THE  SUITABILITY  OF  THIS 
MATERIAL  FOR  ANY  PURPOSE  OR  ABOUT  ANY  OTHER  MATTER,  AND  THIS  MATERIAL  IS  PROVIDED  WITHOUT 
EXPRESS  OR  IMPLIED  WARRANTY  OF  ANY  KIND. 


Appendix  B,  Systems  Engineering  Maturity  and  Benchmarking  Questionnaire,  is  derived,  in  part,  from  A  Systems 
Engineering  Capability  Maturity  Modef^,  Version  1.1,  which  is  Copyright  ©  1995  by  Carnegie  Mellon  University.  This 
work  is  a  collaborative  effort  of  Hughes  Space  and  Communications,  Hughes  Telecommunications  and  Space,  Lockheed 
Martin,  Software  Engineering  Institute,  Software  Productivity  Consortium,  and  Texas  Instruments  Incorporated. 
Permission  to  reproduce  this  product  and  to  prepare  derivative  works  from  this  product  is  granted  royalty-free,  provided 
the  copyright  is  included  with  all  reproductions  and  derivative  work. 
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1.  Introduction  to  the  Questionnaire 


Objective 


Questionnaire 

Foundation 


The  objective  this  Questionnaire  is  threefold:  First,  is  to  gather  a  broad  understanding  of 
the  level  of  maturity  of  the  systems  engineering  process  in  use  in  industry  today. 
Second,  is  to  understand  the  tools  and  methods  used  by  these  sampled  organizations  and 
identify  (if  any)  groupings  (domains).  Third,  investigate  possible  correlation  between 
level  of  maturity,  tools  used,  and  methodologies. 


The  basis  of  the  Questionnaire  is  the  Systems  Engineering  Capability  Maturity  Model 
(SE-CMM),  Version  1.1,  which  was  developed  as  collaborative  effort  among  industry, 
academia  and  government  entities.  The  SE-CMM  is  intended  to  assess  the  maturity  of  an 
organization's  systems  engineering  process.  This  assessment  is  based  upon  18  Process 
Areas  (PAs)  that  cover  engineering,  project  and  organization  categories.  They  are  (in 
alphabetical  order): 

PA  01 :  Analyze  Candidate  Solutions 
PA  02:  Derive  and  Allocate  Requirements 
PA  03:  Evolve  System  Architecture 
PA  04:  Integrate  Disciplines 
PA  05:  Integrate  System 

PA  06:  Understand  Customer  Needs  and  Expectations 
PA  07:  Verify  and  Validate  System 
PA  08:  Ensure  Quality 
PA  09:  Mange  Configurations 
PA  10:  Mange  Risk 

PA  1 1 :  Monitor  and  Control  Technical  Effort 
PA  12:  Plan  Technical  Effort 

PA  13:  Define  Organization's  Systems  Engineering  Process 
PA  14:  Improve  Organization's  Systems  Engineering  Processes 
PA  15:  Manage  Product  Line  Evolution 
PA  16:  Manage  Systems  Engineering  Support  Environment 
PA  17:  Provide  Ongoing  Skills  and  Knowledge 
PA  18:  Coordinate  With  Suppliers 
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1.  Introduction  to  the  Questionnaire 


For  each  process  area,  there  is  an  associated  capability  level  or  maturity  level  identifying 
how  well  an  organization  performs  the  practice.  Table  1-1  shows  the  levels  of  maturity. 

Table  1-1.  Levels  of  Maturity 


Maturity  Capability 
Level  Level 


0  Not  Performed 


1 


Planned  and 
Tracked 


Well  Defined 


Quantitatively 

Controlled 


Continuously 

Improving 


Common  Features 


Base  practices  performed 


Committing  to  perform 
Planning  performance 
Disciplined  performance 
Tracldng  performance 
Verifv  performance 


Defining  a  standard  process 
Tailoring  the  standard  process 
Using  data 

Perform  the  defined  process 


Establishing  measurable  quality  goals 
Determining  process  capability  to  achieve 
goals 

Obiectivelv  managing  performance 


The  Questionnaire  is  structured  around  the  18  process  areas.  For  each  process  area,  a 
common  template  has  been  develop  to  address  the  following  questions 


Process  Adequacy: 

•  Is  the  process  area  generally  performed  to  a  sufficient 
level  to  ensure  a  cost  effective  system  which  satisfies 
the  customers  requirements 

•  What  are  the  limiting  factors  for  the  process  area 

Applied 

Methodology: 

•  What  methods  are  generally  performed  for  the  process 
area 

•  What  tools  are  generally  used 

•  What  internal,  industry  and  government  standards  are 
used 

•  What  metrics  are  used  to  monitor  the  process 

•  What  are  the  two  limiting  items  for  the  process 
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Questionnaire 

Structure 


Estimated  Time 
to  Complete 

Confidentiality 


Frequently 
Asked  Q 

Questions 

A 


Q 

A 


The  Questionnaire  is  made  up  of  four  sections: 

1 .  2.  Organizational  information 

General  information  about  the  respondent  and  the  organization.  This 
information  is  used  for  logistical  purposes  only  and  is  not  contained  in  the 
data  analysis  sample. 

2.  3.  Systems  engineering  overview 

This  section  attempts  to  capture  a  general  overview  of  the  systems 
engineering  process  within  an  organization. 

3.  4.  Key  systems  engineering  process  areas 

This  section  gathers  maturity  data,  tools  and  methods  used  for  each  of  the  18 
process  areas. 

4.  5.  Survey  feedback  form 

This  section  allows  for  the  participant  to  provided  feedback  on  the 
Questionnaire  to  the  writers  for  future  updates. 


4-9  Hours  (15-30  minutes/area  @  18  process  areas  &  systems  engineering  overview) 

The  information  contained  in  this  Questionnaire  is  maintained  in  the  highest  confidence 
and  access  permitted  to  only  Software  Productivity  Consortium's  data  reduction  team 
members.  tFsage  and  publication  of  the  findings  will  be  in  non-attributable  aggregate 
forms  only. 


What  is  a  process  area? 

The  SE-CMM  defines  a  process  area  as  a  "purpose  and  a  set  of  related  systems 
engineering  process  characteristics,  when  performed  collectively,  can  achieve  the  defined 
purpose." 


If  I  don’t  do  a  particular  process  area,  what  should  I  do? 

If  you  do  not  perform  the  process  area,  find  an  appropriate  individual(s)  who  can  address 
the  process  area  and  have  them  fill  it  out.  The  Questionnaire  covers  a  wide  of  range  of 
activities  that  my  be  performed  my  different  individuals.  It  is  perfectly  expectable  to  have 
several  individuals  filling  out  the  various  process  areas.  In  such  a  case,  each  individual 
should  fill-out  a  copy  of  the  Survey  Feedback  form  located  in  the  back  of  the 
Questionnaire. 

If  your  organization  does  not  perform  the  process  area,  you  don't  have  to  fill  out  that 
particular  section.  After  the  description  of  the  process  area,  you  will  be  asked  if  your 
organization  performs  this  process  area,  indicate  a  NOy  and  skip  to  the  next  process  area. 
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1.  Introduction  to  the  Questionnaire 


Q  Has  the  Questionnaire  been  verified  and  validated? 

A  Yes.  The  Questionnaire  is  based  upon  the  industry  available  SE-CMM  and  developed  by 
some  of  the  members  of  the  authoring  team.  Verification  was  done  by  an  internal 
questionnaire  experts  and  validation  by  an  independent  group  of  practicing  systems 
engineers. 


Q  Do  I  need  to  know  the  SE-CMM  or  have  a  copy  of  it  to  fill  out  this  Questionnaire? 
A  No  additional  documents  should  be  required  to  explain  this  Questionnaire, 


Q  Can  you  define  some  of  your  terms? 

A  The  following  definitions  are  from  the  SE-CMM: 

Method:  A  reasonably  complete  set  of  rules  and  criteria  that  establish  a  practice  and 

repeatable  way  of  performing  a  task  to  provide  a  desired  result. 

Methodology:  A  collection  of  methods,  procedures,  and  standards  that  defines  an 

integrated  synthesis  of  engineering  approaches  to  the  development  of  a 
product 

Metric:  A  composite  of  two  or  more  measurements  resulting  is  a  value  that 

defines  a  characteristic  of  the  process 

Organization:  A  unit  within  an  entity  (e.g.,  company,  government  agency,  or  branch 

of  service)  within  which  many  projects  are  managed  as  a  whole.  All 
projects  within  an  organization,  at  the  top  of  the  reporting  structure, 
share  a  common  manager  and  common  policies. 

System:  An  integrated  composite  of  people,  products,  and  processes  that  provide 

a  capability  to  satisfy  a  stated  need  or  objective. 

Systems  The  selective  application  of  scientific  and  engineering  efforts  to: 

Engineering:  1)  transform  an  operational  need  into  a  description  of  a  system 

configuration  which  best  satisfies  the  operational  need 
according  to  the  measures  of  effectiveness; 

2)  integrate  related  technical  parameters  and  ensure  compatibility 
of  all  physical,  functional,  and  technical  program  interfaces  in 
a  manner  which  optimizes  the  total  system  definition  and 
design; 

3)  integrate  the  efforts  of  all  engineering  disciplines  and 
specialties  into  the  total  engineering  effort. 
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Q  Where  can  I  get  a  copy  of  the  SE-CMM? 

A  The  SE-CMM  (Systems  Engineering  Capability  Maturity  Model,  Vl.l  CMU/SEI-95- 
MM-003)  is  available  from  several  sources: 


Software  Productivity  Consortium 
SPC  Building 
2214  Rock  Hill  Road 
Herndon,  Virginia  22070 

1-800-827-4SPC 

Electronic  copy  is  also  available  on¬ 
line.  Please  contact  SPC  for  more 
information. 


Software  Engineering  Institute 
Carnegie  Mellon  University 
Pittsburgh,  Pennsylvania  15213 

412-268-2000 

Electronic  copy  is  also  available  on-line. 
Please  contact  SEI  for  more  information. 


Defense  Technical  Information  Center 
(DTIC) 

Attn:  DTIC-OCP 

8725  John  J.  Kingman  Road 

Suite  0944 

Ft.  Bel  voir,  VA  22060-6218 
703-767-8019/8021/8022/8023 


National  Technical  Information  Service 
(NTIS) 

U.S.  Department  of  Commerce 
Springfield,  V A  22161 

703-487-4600 


Research  Access,  Inc. 

800  Vinial  Street 
Pittsburgh  Pennsylvania  15212 

1-800-685-6510 
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2.  Organizational  Information 


Respondent 

Information 


Organizational 

Information 


Name:  _ Date: 

Title:  _ 

Phone  _ 

Fax  _ 

Project:  _ 


Company 

Division 

Address 


1 .  Who  are  your  organization's  customers?  [Check  all  that  apply] 

a.  [  ]  Department  of  Defense 

b.  [  ]  Civilian  Federal  Government 

c.  [  ]  State  Government 

d.  [  ]  Local  Government 

e.  [  ]  Commercial 

f.  [  ]  Industrial 

2.  Which  best  describes  the  activities  in  which  your  organization  is  involved?  [Check 
all  that  apply] 


a. 

[ 

] 

Light  manufacturing 

h. 

[ 

] 

Automotive 

b. 

[ 

] 

Heavy  manufacturing 

i. 

[ 

] 

Aircraft 

c. 

[ 

] 

Software  development 

j* 

[ 

] 

Communication 

d. 

[ 

] 

Hardware  development 

k. 

[ 

] 

Mining 

e. 

[ 

] 

System  development 

1. 

[ 

] 

Shipping 

f. 

[ 

] 

Engineering  consulting 

m. 

[ 

] 

Building 

g- 

[ 

] 

Other: 

n. 

[ 

] 

Commercial 

What  is  the  value  of  sales  at  your  organization?  [Check  one] 

a. 

[ 

] 

>$1B 

b. 

[ 

] 

$750M  -  $1B 

c. 

[ 

] 

$500M-  $750M 

d 

[ 

] 

$250M-  $500M 

e. 

[ 

] 

$100M  -  $250M 

f. 

[ 

] 

$50M-  $100M 

g- 

[ 

]  <  $50M 

4.  Approximately,  how  many  employees  are  there  at  your  organization's  site? 


continued  on  next  page 
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2.  Organizational  Information 


5.  Based  on  content  of  work,  what  percentage  of  your  organization's  work  is  related  to: 

a.  Hardware  _  d.  Data  _ 

b.  Software _  e.  Facilities _ 

c.  Other  _  _ 

6.  Based  on  your  organization's  annual  expenditures,  what  percentage  of  your 
organization's  operations  are: 

a.  New  system  development  _ _ 

b.  Modifications/upgrades  to  existing  system  _ 

c.  Maintenance  of  existing  systems  _ 

d.  Other 


end  of  2.  Organizational  Information 
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3.  Systems  Engineering  Overview 


1 .  Process 
Maturity 


a.  In  your  opinion,  how  consistently  is  the  systems  engineering  process  performed 
across  your  organization? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  Systems  engineering  process  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 


2 .  Process  a.  In  your  opinion,  does  your  organization  generally  perform  systems  engineering  to  a 

Adequacy  sufficient  level  to  ensure  a  cost  effective  system  which  satisfies  your  customers 

requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Systems  engineering  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=:Always,  D=Don’t  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 

3.  Applied  a.  What  methods  are  generally  use  in  your  systems  engineering  effort? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 

this  questionnaire) 


continued  on  next  page 
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3.  Systems  Engineering  Overview 


b.  What  tools  do  you  generally  use  in  your  systems  engineering  effort? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization’s  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don’t  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  systems  engineering 

Improvement  process  in  your  organization? 


continued  on  next  page 
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3.  Systems  Engineering  Overview 


5,  Systems 
engineering 
practice 


a.  Does  your  organization  have  an  accepted  and  documented  definition  of  systems 
engineering? 

[  ]  Yes 

[  ]  No  (if  not,  go  to  question  c) 

[  ]  Don't  know 


b.  Briefly,  what  is  your  organization's  accepted  and  documented  definition  of  systems 
engineering? 


c.  Is  there  a  separate  group  chartered  to  do  systems  engineering  within  your 
organization? 

[  ]  Yes 

[  ]  No 

[  ]  Don't  know 

d  Which  of  the  following  performs  most  of  the  systems  engineering  in  your 
organization?  [Check  one] 

1.  [  ]  Multi-disciplined  product  teams 

2.  [  ]  Multi-disciplined  systems  engineering  teams 

3 .  [  ]  Systems  engineers  who  are  identified  by  that  job  title 

4.  [  ]  Hardware  engineers 

5.  [  ]  Software  engineers 

6.  [  ]  Project  engineers 

7.  [  ]  Project  managers 

8.  [  ]  Customer 

9.  [  ]  Other 

10.  [  ]  Don't  know/Not  sure 


end  of  3.  Systems  Engineering  Overview 
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3.  Systems  Engineering  Overview 
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Version  1.1.0 


4.  Key  Systems  Engineering  Process  Areas 


Overview  The  following  section  of  the  Questionnaire  contains  the  18  process  areas  (in 

alphabetical  order)  of  systems  engineering  as  described  in  the  Systems  Engineering 
Capability  Maturity  Model  (SE-CMM).  Included  in  the  title  of  each  process  area  is  the 
corresponding  Key  Focus  Area  (KFA)  as  described  in  the  Systems  Engineering 
Capability  Assessment  Model  (SECAM)*. 

Each  process  area  begins  with  a  summary  description  and  a  list  of  base  practices 
questions.  These  questions  are  used  to  address  the  relative  maturity  of  the  organization 
in  the  process  area.  Following  the  base  practices,  is  a  series  of  questions  that  1)  address 
how  effective  you  feel  your  process  is;  2)  what  methodologies,  tools,  standards  and 
metrics  are  used  by  your  organization  in  this  process  area,  and  3)  how  your  processes 
can  be  improved. 


Instructions: 


Step 

Action 

1. 

Read  the  process  area  summary 

2. 

Identify  how  consistently  you  perform  this  process  area 

3. 

If  you  don't  perform  this  process  area,  skip  to  the  next  process  area 

4. 

Identify  the  base  practices  that  are  performed  on  your  project  using  the  scale: 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

5. 

If  you  had  only  N's  (Never),  S’s  (Seldom)  and  D's  (Don’t  know),  skip  to  the 
next  process  area 

6. 

Answer  all  the  questions  that  follow  (parts  2,  3  and  4).  If  you  need  additional 
space,  please  use  the  back  of  the  Questionnaire  of  the  appropriate  process  area 

*  The  SECAM  is  described  in  "Systems  Engineering  Capability  Assessment  Model 
-  Interim  Model  Questionnaire,  Version  1.40";  CAWG- 1995-05- 1.40  and  is 
available  from: 

International  Council  on  Systems  Engineering  (INCOSE) 

2033  Sixth  Avenue,  Suite  804 
Seattle,  WA  98121 

206-441-1164  or  800-366-1164 
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4.  Key  Systems  Engineering  Process  Areas 
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4.  Key  Systems  Engineering  Process  Areas 


PA  01:  Analyze  Candidate  Solutions 

(KFA:  3.4  Integrated  Engineering  Analysis) 


Process  Area  The  purpose  of  Analyze  Candidate  Solutions  is  to  perform  studies  and  analyses  that  will 
Summary  result  in  the  selection  of  a  solution  to  meet  the  identified  problem  and  its  defined 

constraints.  Analyze  Candidate  Solutions  involves  defining  the  approach  and  evaluation 
criteria  for  the  analysis,  as  well  as  for  choosing,  selecting,  and  studying  the  candidate 
solutions.  It  also  involves  communicating  the  rationale  and  results  of  the  analysis. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

Consistency 

1 .  [  ]  Establish  evaluation  criteria  based  on  the  identified  problem  and  its 

defined  constraints. 

2.  [  ]  Define  the  general  approach  for  the  analysis,  based  on  the  established 

evaluation  criteria. 

3.  [  ]  Identify  alternatives  for  evaluation  in  addition  to  those  provided  with  the 

problem  statement. 

4.  [  ]  Analyze  the  competing  candidate  solutions  against  the  established 

evaluation  criteria. 

5.  [  ]  Select  the  solution  that  satisfies  the  established  evaluation  criteria. 

6.  [  ]  Capture  the  disposition  of  each  alternative  under  consideration  and  the 

rationale  for  the  disposition. 


2 .  Process  a.  Does  your  project  generally  Analyze  Candidate  Solutions  to  a  sufficient  level  to 

Adequacy  ensure  a  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Analyze  Candidate  Solutions  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 

continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  01:  Analyze  Candidate  Solutions,  Continued 


3  -  Applied  a.  What  methods  do  you  generally  use  to  Analyze  Candidate  Solutions? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Analyze  Candidate  Solutions? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization’s  performance  for  internal 

use  only 

[  ]  No 
[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Analyze  Candidate 

Improvement  Solutions  practice  in  your  organization? 


end  of  PA  01:  Analyze  Candidate  Solutions 
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4.  Key  Systems  Engineering  Process  Areas 


PA  02:  Derive  and  Allocate  Requirements 

(KFA:  3.2  System  Requirements) 


Process  Area 
Summary 


1 .  Process 
Maturity 


(N=Never,  S=Seldom,  U=UsualIy,  0=0ften,  A=Always,  D=Don't  know) 
Consistency 

1 .  [  ]  Develop  a  detailed  operational  concept  of  the  interaction  of  the  system, 

the  user,  and  the  environment,  that  satisfies  the  operational  need. 

2.  [  ]  Identify  key  requirements  that  have  a  strong  influence  on  cost,  schedule, 

functionality,  risk,  or  performance. 

3 .  [  ]  Partition  requirements  into  groups  based  on  established  criteria  (such  as 

similar  functionality,  performance,  or  coupling)  to  facilitate  and  focus 
the  requirements  analysis. 

4.  [  ]  Derive,  from  the  system  and  other  (e.g.,  environmental)  requirements, 

requirements  that  may  be  logically  inferred  and  implied  as  essential  to 
system  effectiveness. 

5.  [  ]  Identify  the  requirements  associated  with  external  interfaces  to  the 

system  and  interfaces  between  functional  partitions  or  objects. 

6.  [  ]  Allocate  requirements  to  functional  partitions,  objects,  people,  or 

support  elements  to  support  synthesis  of  solutions. 

7.  [  ]  Analyze  requirements  to  ensure  that  they  are  verifiable  by  the  methods 

available  to  the  development  effort. 

8 .  [  ]  Maintain  requirements  traceability  to  ensure  that  lower  level  (derived) 

requirements  are  necessary  and  sufficient  to  meet  the  objectives  of 
higher  level  requirements. 

9.  [  ]  Capture  system  and  other  requirements,  derived  requirements,  derivation 

rationale,  allocations,  traceability,  and  requirements  status. 


The  purpose  of  Derive  and  Allocate  Requirements  is  to  analyze  the  system  and  other 
requirements  and  derive  a  more  detailed  and  precise  set  of  requirements.  These  derived 
requirements  are  allocated  to  system  functions’,  objects',  people',  and  supporting 
processes,  products,  and  services,  which  can  be  used  to  synthesize  solutions.  This 
process  area  addresses  both  the  analysis  of  system-level  requirements  and  the  allocation 
of  system-level  or  derived  requirements  to  lower  level  functions  or  objects.  This 
involves  addressing  the  concept  of  operations,  functional  partitioning,  object 
identification,  and  performance  allocation,  as  well  as  capturing  the  status  and 
traceability  of  requirements.  The  derived  and  allocated  requirements  will  evolve  as  the 
systems  requirements  evolve  over  time.  When  corrective  actions  have  an  impact  on 
requirements,  it  may  be  necessary  to  revise  the  derived  and  allocated  requirements. 


a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  02:  Derive  and  Allocate  Requirements,  Continued 


2 .  Process  a.  Does  your  project  generally  Derive  and  Allocate  Requirements  to  a  sufficient  level  to 

Adequacy  ensure  a  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  Derive  and  Allocate  Requirements  sufficiently  performed 

•) 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

1 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 

3 .  Applied  a.  What  methods  do  you  generally  use  to  Derive  and  Allocate  Requirements? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 

this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Derive  and  Allocate  Requirements? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  02:  Derive  and  Allocate  Requirements,  Continued 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization’s  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don’t  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Derive  and  Allocate 

Improvement  Requirements  practice  in  your  organization? 


end  of  PA  02:  Derive  and  Allocate  Requirements 
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4.  Key  Systems  Engineering  Process  Areas 


PA  03:  Evolve  System  Architecture 

(KFA:  3.3  System  Design) 


Process  Area  The  purpose  of  Evolve  System  Architecture  is  to  provide  a  basis  for  establishing  and 
Summary  evolving  a  system  design.  It  involves  deriving  the  architecture  requirements, 

identifying  key  design  issues,  determining  the  functional  and  physical  structure  and 
interfaces,  and  allocating  the  architecture  requirements  to  system  elements.  The 
practices  described  herein  are  expected  to  be  performed  iteratively  with  other  systems 
engineering  practices  until  the  architecture  is  handed  off  to  the  implementing  or 
component  engineering  disciplines. 

System  architecture  comprises  functional  (or  logical),  physical  (tangible),  and 
foundation  architectures.  Evolve  System  Architecture  activities  are  applicable  to  all 
life-cycle  phases  of  a  product  and  may  be  initiated  either  by  new  development,  changes 
in  requirements,  or  corrective  actions. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

Consistency 

1 .  [  ]  Derive  the  requirements  for  the  system  architecture. 

2.  [  ]  Identify  the  key  design  issues  that  must  be  resolved  to  support 

successful  development  of  the  system. 

3.  [  ]  Generate  altemative(s)  and  constraints  for  the  architecture  and  select  a 

solution  in  accordance  with  the  Analyze  Candidate  Solutions  process 
area  (PA  01). 

4.  [  ]  Develop  the  interface  requirements  for  the  selected  architecture 

components. 

5.  [  ]  Allocate  the  system  and  derived  requirements  to  the  chosen  architecture 

components  and  interfaces. 

6.  [  ]  Maintain  requirement  traceability  for  the  architecture's  requirements  to 

ensure  that  lower  level  (derived)  requirements  are  necessary  and  sufficient 
to  meet  the  needs  of  higher  level  requirements  or  design. 

7.  [  ]  Describe  the  system  architecture  by  capturing  the  design  results  and 

rationale. 

8 .  [  ]  Identify  appropriate  derived  requirements  that  address  the  effectiveness 

and  cost  of  life-cycle  phases  following  development,  such  as  production 
and  operation. 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  03:  Evolve  System  Architecture,  Continued 


2  •  Process  a.  Does  your  project  generally  Evolve  System  Architecture  to  a  sufficient  level  to 

Adequacy  ensure  a  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  Evolve  System  Architecture  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

I 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 

3 .  Applied  a.  What  methods  do  you  generally  use  to  Evolve  the  System  Architecture? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Evolve  the  System  Architecture? 
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4.  Key  Systems  Engineering  Process  Areas 


PA  03:  Evolve  System  Architecture,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization’s  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Evolve  System 

Improvement  Architecture  practice  in  your  organization? 


end  of  PA  03:  Evolve  System  Architecture 
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4.  Key  Systems  Engineering  Process  Areas 


PA  04:  Integrate  Disciplines 

(KFA:  1.4  Intergroup  Coordination) 


Process  Area  The  purpose  of  Integrate  Disciplines  is  to  identify  those  disciplines  necessary  for 
Summary  effective  system  development  and  create  an  environment  in  which  they  jointly  and 

effectively  work  together  toward  a  common  agenda.  Each  discipline’s  unique  expertise 
and  concerns  are  brought  forward  and  considered,  but  the  focus  on  total  system 
development  is  maintained.  These  disciplines  may  include,  but  are  not  limited  to, 
problem  domain,  marketing,  manufacturing,  component  design,  development, 
reliability,  maintainability,  operations,  quality,  supportability,  human  factors, 
logistics,  safety,  and  security.  It  is  critical  to  be  able  to  meld  such  disciplines  without 
sacrificing  their  parochial  interests  concerning  issues  important  to  and  unique  to  each 
discipline.  This  cooperative  environment  must  persist  throughout  the  system  life 
cycle. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A= Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

Consistency 

1 .  [  ]  Involve  the  disciplines  that  are  essential  to  system  development  in  a 

timely  manner. 

2.  [  ]  Promote  cross-discipline  understanding  among  the  developers. 

3.  [  ]  Establish  methods  for  interdisciplinary  coordination. 

4.  [  ]  Establish  and  use  methods  for  identifying  and  resolving  interdisciplinary 

issues,  and  creating  integrated  solutions. 

5.  [  ]  Communicate  results  of  interdisciplinary  activities  to  affected  groups. 

6.  [  ]  Develop  project  goals  and  ensure  that  all  affected  groups  and  individuals 

are  fully  aware  of  them. 


continued  on  next  page 
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PA  04:  Integrate  Disciplines,  Continued 


2 .  Process  a.  Does  your  project  generally  Integrate  Disciplines  to  a  sufficient  level  to  ensure  a 

Adequacy  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Integrate  Disciplines  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Budget  limitations 
Schedules  limitations 
Policies/procedures 
Methodologies/practices 
Tool  availability 
Tool  adequacy 

Systems  engineering  personnel  availability 
Engineering  personnel  availability 
Training  and  personnel  experience 
Other 


3 .  Applied  a.  What  methods  do  you  generally  use  to  Integrate  Disciplines? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Integrate  Disciplines? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  04:  Integrate  Disciplines,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Integrate  Disciplines 

Improvement  practice  in  your  organization? 


end  of  PA  04:  Integrate  Disciplines 
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4^  Key  Systems  Engineering  Process  Areas 


PA  05:  Integrate  System 

(KFA:  3.5  System  Integration) 


Process  Area  The  purpose  of  Integrate  System  is  to  ensure  that  system  elements  will  function  as  a 

Summary  whole.  This  primarily  involves  identifying,  defining,  and  controlling  interfaces,  as  well 

as  verifying  system  functions  that  require  multiple  system  elements.  The  activities 
associated  with  Integrate  System  occur  throughout  the  entire  product  life  cycle. 

1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U-Usually,  0=0ften,  A=Always,  D=Don't  know) 


Develop  detailed  specifications  of  the  interfaces  implied  by  the  system 
architecture. 

Coordinate  interface  specifications  and  changes  with  all  affected  groups 
and  individuals. 

Verify  the  receipt  of  each  system  element  required  to  assemble  the 
system  in  accordance  with  the  physical  architecture. 

Verify  the  implemented  design  features  of  developed  or  purchased 
system  elements  against  their  requirements. 

Verify  that  the  system  element  interfaces  comply  with  the  interface 
specifications  prior  to  assembly. 

Assemble  aggregates  of  system  elements  in  accordance  with  the 
established  integration  strategy. 

Check  the  integrated  system  interfaces  in  accordance  with  the  established 
integration  strategy. 

Develop  an  integration  strategy  and  supporting  documentation  that 
identify  the  optimal  sequence  for  receipt,  assembly,  and  activation  of  the 
various  components  that  make  up  the  system. 

continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  05:  Integrate  System,  Continued 


2 .  Process  a.  Does  your  project  generally  Integrate  System  to  a  sufficient  level  to  ensure  a  cost 

Adequacy  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Integrate  System  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

1 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 

3  •  Applied  a.  What  methods  do  you  generally  use  to  Integrate  System? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Integrate  System? 


continued  on  next  page 


30 


Version  1.1.0 


4.  Key  Systems  Engineering  Process  Areas 


PA  05:  Integrate  System,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization’s  performance  for  internal 

use  only 

[  ]  No 
[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Integrate  System 

Improvement  practice  in  your  organization? 


end  of  PA  05:  Integrate  System 
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4.  Key  Systems  Engineering  Process  Areas 


PA  06:  Understand  Customer  Needs  and  Expectations 

(KFA:  3.1  System  Concept  Definition) 


Process  Area  The  purpose  of  Understand  Customer  Needs  and  Expectations  is  to  elicit,  stimulate,  analyze, 

Summary  and  communicate  customer  needs  and  expectations  to  obtain  a  better  understanding  of  what 

will  satisfy  the  customer.  Understand  Customer  Needs  and  Expectations  involves  engaging 
the  customer  or  surrogate  in  ongoing  dialogue  designed  to  translate  his/her  needs  and 
expectations  into  a  verifiable  set  of  requirements  which  the  customer  understands  and  which 
provide  the  basis  for  agreements  between  the  customer  and  the  systems  engineering  effort. 

Customer  needs  typically  change  over  time.  Organizations  need  to  have  a  workable  way  to 
incorporate  such  changes  into  current  and  future  version  of  the  product. 


1  •  Process 
Maturity 


a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 


b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Consistency 

1.  [  ] 

2.  [  ] 

3.  [  ] 

4.  [  ] 

5.  [  ] 


Elicit  the  customer's  needs,  expectations,  &  measures  of  effectiveness. 
Analyze  the  customer's  needs  and  expectations  to  develop  a  preliminary 
operational  concept  of  the  system. 

Develop  a  statement  of  system  requirements. 

Obtain  the  customers'  agreement  that  system  requirements  satisfy  their 
needs  and  expectations. 

Inform  the  customer  on  a  regular  basis  about  the  status  and  disposition  of 
needs,  expectations,  and  measures  of  effectiveness. 


2 .  Process  a.  Does  your  project  generally  Understand  the  Customer  Needs  and  Expectations  to  a 

Adequacy  sufficient  level  to  ensure  a  cost  effective  system  which  satisfies  your  customers 

requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=Often,  A=Always,  D=Don't  know) 

[  ]  Understand  Customer  Needs  and  Expectations  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 

continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  06:  Understand  Customer  Needs  and  Expectations,  Continued 


3  •  Applied  a.  What  methods  do  you  generally  use  to  Understand  Customer  Needs  and 

Methodology  Expectations?  (Please  describe,  if  required,  please  use  additional  paper  and  attach  to 
the  end  of  this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Understand  Customer  Needs  and  Expectations? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  “  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 
[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Understand  Customer 

Improvement  Needs  and  Expectations  practice  in  your  organization? 


end  of  PA  06:  Understand  Customer  Needs  and  Expectations 
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4.  Key  Systems  Engineering  Process  Areas 


PA  07:  Verify  and  Validate  System 
(KFA:  3.6  System  Verification) 
(KFA:  3.7  System  Validation) 


Process  Area  The  purpose  of  Verify  and  Validate  System  is  to  ensure  that  the  developer/supplier  team 
Summary  performs  increasingly  comprehensive  evaluations  to  ensure  that  evolving  work  products 

will  meet  all  requirements.  The  activities  associated  with  Verify  and  Validate  System 
begin  early  in  the  development,  address  all  work  products  (including  requirements  and 
design),  and  continue  through  development  and  integration  of  system  elements  into 
production,  use,  and  disposal  of  the  system.  The  scope  of  verification  covers 
development  of  the  full  system,  as  well  as  its  production,  operation,  and  support. 
Validation  is  a  measure  of  customer  satisfaction,  given  the  customer’s  operational  need. 
Validation  should  continue  throughout  product  use. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

Consistency 

1 .  [  ]  Establish  plans  for  verification  and  validation  that  identify  the  overall 

requirements,  objectives,  resources,  facilities,  special  equipment,  and 
schedule  applicable  to  the  system  development, 

2.  [  ]  Define  the  methods,  process,  reviews,  inspections,  and  tests  by  which 

incremental  products  that  were  verified  against  established  criteria  or 
requirements  that  were  established  in  a  previous  phase. 

3.  [  ]  Define  the  methods,  process,  and  evaluation  criteria  by  which  the 

system  or  product  is  verified  against  the  system  or  product  requirements. 

4.  [  ]  Define  the  methods,  process,  and  evaluation  criteria  by  which  the 

system  or  product  will  be  validated  against  the  customer’s  needs  and 
expectations. 

5 .  [  ]  Perform  the  verification  and  validation  activities  that  are  specified  by  the 

verification  and  validation  plans  and  procedures,  and  capture  the  results. 

6.  [  ]  Compare  the  collected  test,  inspection,  or  review  results  with 

established  evaluation  criteria  to  assess  the  degree  of  success. 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  07:  Verify  and  Validate  System,  Continued 


2  •  Process  a.  Does  your  project  generally  Verify  and  Validate  Systems  to  a  sufficient  level  to 

Adequacy  ensure  a  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Verify  and  Validate  System  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

Impact 

1.  [  ]  Budget  limitations 

2.  [  ]  Schedules  limitations 

3.  [  ]  Policies/procedures 

4.  [  ]  Methodologies/practices 

5.  [  ]  Tool  availability 

6.  [  ]  Tool  adequacy 

7.  [  ]  Systems  engineering  personnel  availability 

8 .  [  ]  Engineering  personnel  availability 

9.  [  ]  Training  and  personnel  experience 

10.  [  ]  Other  _ 


3 .  Applied  a.  What  methods  do  you  generally  use  to  Verify  and  Validate  System? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Verify  and  Validate  System? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  07:  Verify  and  Validate  System,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Verify  and  Validate 

Improvement  System  practice  in  your  organization? 


end  of  PA  07:  Verify  and  Validate  System 
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4.  Key  Systems  Engineering  Process  Areas 
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4.  Key  Systems  Engineering  Process  Areas 


PA  08:  Ensure  Quality 

(KFA:  1.6  Quality  Assurance) 


Process  Area  The  purpose  of  Ensure  Quality  is  to  address  not  only  the  quality  of  the  system,  but  also 

Summary  the  quality  of  the  process  being  used  to  create  the  system  and  the  degree  to  which  the 

project  follows  the  defined  process.  The  underlying  concept  of  this  process  area  is  that 
high-quality  systems  can  only  be  consistently  produced  on  a  continuous  basis  if  a 
process  exists  to  continuously  measure  and  improve  quality.  In  addition,  this  process 
must  be  adhered  to  rigorously  and  throughout  the  system  life  cycle.  Key  aspects  of  the 
process  required  to  develop  high-quality  systems  are  measurement,  analysis,  and 
corrective  action. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

Consistency 

1 .  [  ]  Ensure  the  defined  system  engineering  process  is  adhered  to  during  the 

system  life  cycle. 

2.  [  ]  Evaluate  work  product  measures  against  the  requirements  for  work 

product  quality. 

3.  [  ]  Measure  the  quality  of  the  systems  engineering  process  used  by  the 

project. 

4.  [  ]  Analyze  quality  measurements  to  develop  recommendations  for  quality 

improvement  or  corrective  action  as  appropriate. 

5.  [  ]  Obtain  employee  participation  in  identifying  and  reporting  quality 

issues. 

6.  Initiate  activities  that  address  identified  quality  issues  or  quality 
improvement  opportunities. 

7.  [  ]  Establish  a  mechanism  or  a  set  of  mechanisms  to  detect  the  need  for 

corrective  actions  to  processes  or  products. 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  08:  Ensure  Quality,  Continued 


2 .  Process  a.  Does  your  project  generally  Ensures  Quality  to  a  sufficient  level  to  ensure  a  cost 

Adequacy  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=:0ften,  A=Always,  D=Don't  know) 

[  ]  Ensure  Quality  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 

3  -  Applied  a.  What  methods  do  you  generally  use  to  Ensure  Quality? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Ensure  Quality? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  08:  Ensure  Quality,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization’s  performance  for  internal 

use  only 

[  ]  No 
[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Ensure  Quality 

Improvement  practice  in  your  organization? 


end  of  PA  08:  Ensure  Quality 
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4.  Key  Systems  Engineering  Process  Areas 
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4,  Key  Systems  Engineering  Process  Areas 


PA  09:  Manage  Configurations 

(KFA:  1.5  Configuration  Management) 


Process  Area  The  purpose  of  Manage  Configurations  is  to  maintain  data  on  and  status  of  identified 

Summary  configuration  units,  and  to  analyze  and  control  changes  to  the  system  and  its 

configuration  units.  Managing  the  system  configuration  involves  providing  accurate 
and  current  configuration  data  and  status  to  developers  and  customers. 

This  process  area  is  applicable  to  all  work  products  that  are  placed  under  configuration 
management.  An  example  set  of  work  products  that  may  be  placed  under  configuration 
management  could  include  hardware  and  software  configuration  items,  design  rationale, 
requirements,  product  data  files,  or  trade  studies. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

Consistency 

1 .  [  ]  Decide  among  candidate  methods  for  configuration  management. 

2.  [  ]  Identify  configuration  units  that  constitute  identified  baselines. 

3.  [  ]  Maintain  a  repository  of  work  product  baselines. 

4.  [  ]  Control  changes  to  established  configuration  units. 

5.  [  ]  Communicate  status  of  configuration  data,  proposed  changes,  and  access 

information  to  affected  groups. 


2 .  Process  a.  Does  your  project  generally  Manage  the  Configuration  to  a  sufficient  level  to  ensure 

Adequacy  a  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Manage  Configurations  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 
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4.  Key  Systems  Engineering  Process  Areas 


PA  09:  Manage  Configurations,  Continued 


3 .  Applied  a.  What  methods  do  you  generally  use  to  Manage  Configurations? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 

this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Manage  Configurations? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Manage 

Improvement  Configurations  practice  in  your  organization? 


end  of  PA  09:  Manage  Configurations 
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4.  Key  Systems  Engineering  Process  Areas 


PA  10:  Manage  Risk 

(KFA:  1.7  Risk  Management) 


Process  Area  The  purpose  of  Manage  Risk  is  to  identify,  assess,  monitor,  and  mitigate  risks  to  the 
Summary  success  of  both  the  systems  engineering  activities  and  the  overall  technical  effort.  This 

process  area  continues  throughout  the  life  of  the  project.  Similar  to  the  Plan  Technical 
Effort  (PA  12)  and  Monitor  and  Control  Technics  Effort  (PA  11)  process  areas,  the  scope  of 
this  process  area  includes  both  the  systems  engineering  activities  and  the  overall  technical 
project  effort,  as  the  systems  engineering  effort  on  the  project  cannot  be  considered 
successful  unless  the  overall  technical  effort  is  successful. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don‘t  know) 

Consistency 

1 .  [  ]  Develop  a  plan  for  risk-management  activities  that  is  the  basis  for 

identifying,  assessing,  mitigating,  and  monitoring  risks  for  the  life  of  the 
project. 

2.  [  ]  Identify  project  risks  by  examining  project  objectives  with  respect  to  the 

alternatives  and  constraints,  and  identifying  what  can  go  wrong. 

3.  [  ]  Assess  risks  and  determine  the  probability  of  occurrence  and  consequence  of 

realization. 

4.  [  ]  Obtain  formal  recognition  of  the  project  risk  assessment. 

5.  [  ]  Implement  the  risk-mitigation  activities. 

6.  [  ]  Monitor  risk-mitigation  activities  to  ensure  that  the  desired  results  are  being 

obtained. 


2  •  Process  a.  Does  your  project  generally  Manages  Risk  to  a  sufficient  level  to  ensure  a  cost 

Adequacy  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Manage  Risk  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

1 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 
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4.  Key  Systems  Engineering  Process  Areas 


PA  10:  Manage  Risk,  Continued 


3 .  Applied  a.  What  methods  do  you  generally  use  to  Manage  Risk? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Manage  Risk? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Manage  Risk  practice 

Improvement  in  your  organization? 


end  of  PA  10:  Manage  Risk 
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4.  Key  Systems  Engineering  Process  Areas 


PA  11:  Monitor  and  Control  Technical  Effort 

(KFA:  1.2  Tracking  and  Oversight) 


Process  Area  The  purpose  of  Monitor  and  Control  Technical  Effort  is  to  provide  adequate  visibility  of 
Summary  actual  progress  and  risks.  Visibility  encourages  timely  corrective  action  when  performance 

deviates  significantly  from  plans. 

Monitor  and  Control  Technical  Effort  involves  directing,  tracking  and  reviewing  the 
project's  accomplishments,  results,  and  risks  against  its  documented  estimates, 
commitments,  and  plans.  A  documented  plan  is  used  as  the  basis  for  tracking  the  activities 
and  risks,  communicating  status,  and  revising  plans. 


1  •  Process 
Maturity 


a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 


b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Consistency 
E  [  ] 

2.  [  ] 

3.  [  ] 

4.  [  ] 

5.  [  ] 

6.  [  ] 


Direct  technical  effort  in  accordance  with  technical  management  plans. 
Track  actual  use  of  resources  against  technical  management  plans. 
Track  performance  against  the  established  technical  parameters. 
Review  performance  against  the  technical  management  plans. 

Analyze  issues  resulting  from  the  tracking  and  review  of  technical 
parameters  to  determine  corrective  actions. 

Take  corrective  actions  when  actual  results  deviate  from  plans. 


2 .  Process  a.  Does  your  project  generally  Monitor  and  Control  the  Technical  Effort  to  a  sufficient 

Adequacy  level  to  ensure  a  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Monitor  and  Control  Technical  Effort  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A= Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10. 

[ 

] 

Other 
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4.  Key  Systems  Engineering  Process  Areas 


PA  11:  Monitor  and  Control  Technical  Effort,  Continued 


3 .  Applied  a.  What  methods  do  you  generally  use  to  Monitor  and  Control  Technical  Effort? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Monitor  and  Control  Technical  Effort? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  ’  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Monitor  and  Control 

Improvement  Technical  Effort  practice  in  your  organization? 


end  of  PA  11:  Monitor  and  Control  Technical  Effort 


48 


Version  Ll.O 


4.  Key  Systems  Engineering  Process  Areas 


PA  12:  Plan  Technical  Effort 

(KFA:  1.1  Planning) 


Process  Area  The  purpose  of  Plan  Technical  Effort  is  to  establish  plans  that  provide  the  basis  for 
Summary  scheduling,  costing,  controlling,  tracking,  and  negotiating  the  nature  and  scope  of  the 

technical  work  involved  in  system  development,  manufacturing,  use,  and  disposal 
System  engineering  activities  must  be  integrated  into  comprehensive  technical  planning 
for  the  entire  project. 

Plan  technical  effort  involves  developing  estimates  for  the  work  to  be  performed, 
obtaining  necessary  commitments  from  interfacing  groups,  and  defining  the  plan  to 
perform  the  work. 


1  •  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  peiformed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

Consistency 

1 .  [  ]  Identify  resources  that  are  critical  to  the  technical  success  of  the  project. 

2.  [  ]  Develop  estimates  for  the  factors  that  affect  the  magnitude  and  technical 

feasibility  of  the  project. 

3.  [  ]  Develop  cost  estimates  for  all  technical  resources  required  by  the 

project. 

4.  [  ]  Determine  the  technical  process  to  be  used  on  the  project. 

5.  [  ]  Identify  technical  activities  for  the  entire  life  cycle  of  the  project. 

6.  [  ]  Define  specific  processes  to  support  effective  interaction  with  the 

customer(s)  and  supplier(s). 

7.  [  ]  Develop  technical  schedules  for  the  entire  project  life  cycle. 

8.  [  ]  Establish  technical  parameters  with  thresholds  for  the  project  and  the 

system. 

9.  [  ]  Use  the  information  gathered  in  planning  activities  to  develop  technical 

management  plans  that  will  serve  as  the  basis  for  tracking  the  salient 
aspects  of  the  project  and  the  systems  engineering  effort. 

10.  [  ]  Review  the  technical  management  plans  with  all  affected  groups  and 

individuals,  and  obtain  group  commitment. 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  12:  Plan  Technical  Effort,  Continued 


2.  Process 
Adequacy 


a.  Does  your  project  generally  Plan  the  Technical  Effort  to  a  sufficient  level  to  ensure  a 
cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Plan  Technical  Effort  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Impact 

1. 

[ 

]  Budget  limitations 

2. 

[ 

]  Schedules  limitations 

3. 

[ 

]  Policies/procedures 

4. 

•[ 

]  Methodologies/practices 

5. 

[ 

]  Tool  availability 

6. 

[ 

]  Tool  adequacy 

7. 

[ 

]  Systems  engineering  personnel  availability 

8. 

[ 

]  Engineering  personnel  availability 

9. 

[ 

]  Training  and  personnel  experience 

10. 

[ 

]  Other 

3 .  Applied  a.  What  methods  do  you  generally  use  to  Plan  Technical  Effort? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Plan  Technical  Effort? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  12:  Plan  Technical  Effort,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  “  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  “  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Plan  Technical  Effort 

Improvement  practice  in  your  organization? 


end  of  PA  12:  Plan  Technical  Effort 
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4>  Key  Systems  Engineering  Process  Areas 


This  page  intentionally  left  blank. 
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4.  Key  Systems  Engineering  Process  Areas 


PA  13:  Define  Organization’s  Systems  Engineering  Process 

(KFA:  2.1  Process  Management  and  Improvement) 


Process  Area  The  purpose  of  Define  Organization's  Systems  Engineering  Process  is  to  create  and 
Summary  manage  the  organization’s  standard  systems  engineering  processes,  which  can 

subsequently  be  tailored  by  a  project  to  form  the  unique  processes  that  it  will  follow  in 
developing  its  systems  or  products. 

Define  Organization's  Systems  Engineering  Process  involves  defining,  collecting,  and 
maintaining  the  process  that  will  meet  the  business  goals  of  the  organization,  as  well 
as  designing,  developing,  and  documenting  systems-engineering  process  assets.  Assets 
include  example  processes,  process  fragments,  process-related  documentation,  process 
architectures,  process-tailoring  rules  and  tools,  and  process  measurements. 

1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  peiformed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


]  Establish  goals  for  the  organization’s  systems  engineering  process  from 
the  organization's  business  goals. 

]  Collect  and  maintain  systems-engineering  process  assets. 

]  Develop  a  well-defined  standard  systems  engineering  process  for  the 
organization. 

]  Define  guidelines  for  tailoring  the  organization’s  standard  systems 
engineering  process  for  project  use  in  developing  the  project's  defined 
process. 

continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas _ 

PA  13:  Define  Organization's  Systems  Engineering  Process,  Continued 


2 .  Process  a.  Does  your  organization/project  generally  Define  the  Organization’s  Systems 

Adequacy  Engineering  Process  to  a  sufficient  level  to  ensure  a  cost  effective  system  which 

satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  Define  Organization's  Systems  Engineering  Process  sufficiently 

performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10: 

[ 

] 

Other 

3 .  Applied  a.  What  methods  do  you  generally  use  to  Define  Organization's  Systems  Engineering 

Methodology  Process? 

(Please  describe,  if  required,  please  use  additional  paper  and  attach  to  the  end  of  this 
questionnaire) 


b.  What  tools  do  you  generally  use  to  Define  Organization's  Systems  Engineering 
Process? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  13:  Define  Organization’s  Systems  Engineering  Process,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  "  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don’t  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Define  Organization's 

Improvement  Systems  Engineering  Process  practice  in  your  organization? 


end  of  PA  13:  Define  Organization's  Systems  Engineering  Process 
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4.  Key  Systems  Engineering  Process  Areas 


PA  14:  Improve  Organization's  Systems  Engineering  Processes 

(KFA:  2.1  Process  Management  and  Improvement) 


Process  Area  The  purpose  of  Improve  Organization’s  Systems  Engineering  Processes  is  to  gain 
Summary  competitive  advantage  by  continuously  improving  the  effectiveness  and  efficiency  of  the 

systems  engineering  processes  used  by  the  organization.  It  involves  developing  an 
understanding  of  the  organization’s  processes  in  the  context  of  the  organization's 
business  goals,  analyzing  the  performance  of  the  processes,  and  explicitly  planning  and 
deploying  improvements  to  those  processes. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=Often,  A=Always,  D=Don't  know) 

Consistency 

L  [  ]  Appraise  the  existing  processes  being  performed  in  the  organization  to 

understand  their  strengths  and  weaknesses. 

2.  [  ]  Plan  improvements  to  the  organization's  processes  based  on  analyzing 

the  impact  of  potential  improvements  on  achieving  the  goals  of  the 
processes. 

3 .  [  ]  Change  the  organization's  standard  systems  engineering  process  to 

reflect  targeted  improvements. 

4.  [  ]  Communicate  process  improvements  to  existing  projects  and  to  other 

affected  groups,  as  appropriate. 


2 .  Process  a.  Does  your  project/organization  generally  Improve  the  Organization’s  Systems 

Adequacy  Engineering  Processes  to  a  sufficient  level  to  ensure  a  cost  effective  system  which 

satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Improve  Organization’s  Systems  Engineering  Processes 

sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10: 

[ 

] 

Other 
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4.  Key  Systems  Engineering  Process  Areas 


PA  14:  Improve  Organization's  Systems  Engineering  Process,  Continued 


3  •  Applied  a.  What  methods  do  you  generally  use  to  Improve  Organization's  Systems 

Methodology  Engineering  Processes? 

(Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Improve  Organization's  Systems  Engineering 
Processes? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


continued  on  next  page 
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4.  Key  Systems  Engineering  Process  Areas 


PA  15:  Manage  Product  Line  Evolution 

(KFA:  2.3  Technology  Management) 


Process  Area  The  purpose  of  Manage  Product  Line  Evolution  is  to  introduce  services,  equipment,  and  new 
Summary  technology  to  achieve  the  optimal  benefits  in  product  evolution,  cost,  schedule,  and 

performance  over  time  as  the  product  line  evolves  toward  its  ultimate  objectives. 

An  organization  must  first  determine  the  evolution  of  a  product.  Then  the  organization  has 
to  decide  how  it  will  design  and  build  those  products  including  critical  components,  cost- 
effective  tools,  and  efficient  and  effective  processes. 

1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S-Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

r 

]  Define  the  types  of  products  to  be  offered. 

]  Identify  new  product  technologies  or  enabling  infrastructure  that  will  help 
the  organization  acquire,  develop,  and  apply  technology  for  competitive 
advantage. 

]  Make  the  necessary  changes  in  the  product  development  cycle  to  support  the 
development  of  new  products. 

]  Ensure  critical  components  are  available  to  support  planned  product 
evolution. 

]  Insert  new  technology  into  product  development,  marketing,  and 
manufacturing. 


Consistenc; 
1.  [ 

2.  [ 

3.  [ 

4.  [ 

5.  [ 


2 .  Process  a.  Does  your  project/organization  generally  Manage  the  Product  Line  Evolution  to  a 

Adequacy  sufficient  level  to  ensure  a  cost  effective  system  which  satisfies  your  customers 

requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Manage  Product  Line  Evolution  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


1.  [  ]  Budget  limitations 

2.  [  ]  Schedules  limitations 

3.  [  ]  Policies/procedures 

4.  [  ]  Methodologies/practices 

5.  [  ]  Tool  availability 

6.  [  ]  Tool  adequacy 

7.  [  ]  Systems  engineering  personnel  availability 

8.  [  ]  Engineering  personnel  availability 

9.  [  ]  Training  and  personnel  experience 

10:  [  ]  Other  _ 
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4.  Key  Systems  Engineering  Process  Areas 


PA  15:  Manage  Product  Line  Evolution,  Continued 


3  •  Applied  a.  What  methods  do  you  generally  use  to  Manage  Product  Line  Evolution? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 

this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Manage  Product  Line  Evolution? 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 
[  ]  Don’t  know 


4  •  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Manage  Product  Line 

Improvement  Evolution  practice  in  your  organization? 


end  of  PA  15:  Manage  Product  Line  Evolution 
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4.  Key  Systems  Engineering  Process  Areas 


PA  16:  Manage  Systems  Engineering  Support  Environment 

(KFA:  2.4  Environment  and  Tool  Support) 


Process  Area  The  purpose  of  Manage  Systems  Engineering  Support  Environment  is  to  provide  the 

Summary  technology  environment  needed  to  develop  the  product  and  perform  the  process. 

Development  and  process  technology  is  inserted  into  the  environment  with  a  goal  of 
minimizing  disruption  of  development  activities  while  upgrading  to  make  new 
technology  available. 

The  technology  needs  of  an  organization  change  over  time,  and  the  efforts  described  in 
this  process  area  must  be  re-executed  as  the  needs  evolve. 


1 .  Process  a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 

Maturity  (N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

Consistency 

1 .  [  ]  Maintain  awareness  of  the  technologies  that  support  the  organization’s 

goals. 

2.  [  ]  Determine  requirements  for  the  organization’s  systems  engineering 

support  environment  based  on  organizational  needs. 

3.  [  ]  Obtain  a  systems  engineering  support  environment  that  meets  the 

requirements  established  in  Determine  Support  Requirements  by  using 
the  practices  in  the  Analyze  Candidate  Solutions  process  area. 

4.  [  ]  Tailor  the  systems  engineering  support  environment  to  individual 

project’s  needs. 

5.  [  ]  Insert  new  technologies  into  the  systems  engineering  support 

environment  based  on  the  organization's  business  goals  and  the  projects’ 
needs. 

6.  [  ]  Maintain  the  systems  engineering  support  environment  to  continuously 

support  the  projects  dependent  on  it. 

7.  [  ]  Monitor  the  systems  engineering  support  environment  for  improvement 

opportunities. 
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4.  Key  Systems  Engineering  Process  Areas 


PA  16:  Manage  Systems  Engineering  Support  Environment,  Continued 


2 .  Process  a.  Does  your  project/organization  generally  Manage  the  Systems  Engineering  Support 

Adequacy  Environment  to  a  sufficient  level  to  ensure  a  cost  effective  system  which  satisfies 

your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  Manage  Systems  Engineering  Support  Environment  sufficiently 

performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don‘t  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

1 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10: 

[ 

] 

Other 

3 .  Applied  a.  What  methods  do  you  generally  use  to  Manage  Systems  Engineering  Support 

Methodology  Environment? 

(Please  describe,  if  required,  please  use  additional  paper  and  attach  to  the  end  of  this 
questionnaire) 


b.  What  tools  do  you  generally  use  to  Manage  Systems  Engineering  Support 
Environment? 
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4.  Key  Systems  Engineering  Process  Areas 


PA  16:  Manage  Systems  Engineering  Support  Environment,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don't  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Manage  Systems 

Improvement  Engineering  Support  Environment  practice  in  your  organization? 


end  of  PA  16:  Manage  Systems  Engineering  Support  Environment 
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4.  Key  Systems  Engineering  Process  Areas 


PA  17:  Provide  Ongoing  Skills  and  Knowledge 

(KFA:  2.2  Training) 


Process  Area  The  purpose  of  Provide  Ongoing  Skills  and  Knowledge  is  to  ensure  that  projects  and 
Summary  the  organization  have  the  necessary  knowledge  and  skills  to  achieve  project  and 

organizational  objectives.  To  ensure  the  effective  application  of  these  critical  resources 
that  are  predominantly  available  only  from  people,  Ae  knowledge  and  skill  requirements 
within  the  organization  need  to  be  identified,  as  well  as  the  specific  project's  or 
organization's  needs  (such  as  those  relating  to  emergent  programs  or  technology,  and 
new  products,  processes,  and  policies). 

Needed  skills  and  knowledge  can  be  provided  both  by  training  within  the  organization 
and  by  timely  acquisition  from  sources  external  to  the  organization.  Acquisition  from 
external  sources  may  include  customer  resources,  temporary  hires,  new  hires, 
consultants,  and  subcontractors.  In  addition,  knowledge  may  be  acquired  from  subject 
matter  experts. 

а.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don*t  know) 

[  ]  This  process  area  is  consistently  performed 

(if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

Consistency 

1 .  [  ]  Identify  needed  improvements  in  skill  and  knowledge  throughout  the 

organization  using  the  projects'  needs,  organizational  strategic  plan,  and 
existing  employee  skills  as  guidance. 

2.  [  ]  Evaluate  and  select  the  appropriate  mode  of  acquiring  knowledge  or 

skills  with  respect  to  training  or  other  sources. 

3.  [  ]  Ensure  that  appropriate  skill  and  knowledge  are  available  to  the  systems 

engineering  effort. 

4.  [  ]  Prepare  training  materials  based  upon  the  identified  training  needs. 

5.  [  ]  Train  personnel  to  have  the  skills  and  knowledge  needed  to  perform  their 

assigned  roles. 

б.  [  ]  Assess  the  effectiveness  of  the  training  to  meet  the  identified  training 

needs. 

7.  [  ]  Maintain  records  of  training  and  experience. 

8.  [  ]  Maintain  training  materials  in  an  accessible  repository. 

continued  on  next  page 
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4,  Key  Systems  Engineering  Process  Areas 


PA  17:  Provide  Ongoing  Skills  and  Knowledge,  Continued 


2 .  Process  a.  Does  your  project/organization  generally  Provide  the  Ongoing  Skills  and  Knowledge 

Adequacy  to  a  sufficient  level  to  ensure  a  cost  effective  systenn  which  satisfies  your  customers 

requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Provide  Ongoing  Skills  and  Knowledge  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Impact 


1. 

[ 

] 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

] 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10: 

[ 

] 

Other 

3 .  Applied  a.  What  methods  do  you  generally  use  to  Provide  Ongoing  Skills  and  Knowledge? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Provide  Ongoing  Skills  and  Knowledge? 
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4.  Key  Systems  Enjgineerin^  Process  Areas 


PA  17:  Provide  Ongoing  Skills  and  Knowledge,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization's  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization’s  performance  for  internal 

use  only 

[  ]  No 
[  ]  Don’t  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Provide  Ongoing 

Improvement  Skills  and  Knowledge  practice  in  your  organization? 


end  of  PA  17:  Provide  Ongoing  Skills  and  Knowledge 
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4.  Key  Systems  Engineering  Process  Areas 


PA  18:  Coordinate  With  Suppliers 

(KFA:  1.3  Subcontract  Management) 


Process  Area  The  purpose  of  Coordinate  With  Suppliers  is  to  address  the  needs  of  organizations  to 
Summary  effectively  manage  the  portions  of  product  work  that  are  conducted  by  other 

organizations.  Decisions  made  as  a  part  of  this  process  area  should  be  made  in 
accordance  with  the  Analyze  Candidate  Solutions  process  area  (PA  01).  The  general 
term  supplier  is  used  to  identify  an  organization  that  develops,  manufactures,  tests, 
supports,  etc.,  a  component  of  the  system.  Suppliers  may  take  the  form  of  vendors, 
subcontractors,  partnerships,  etc.,  as  the  business  organization  warrants. 

In  addition  to  coordination  of  schedules,  processes,  and  deliveries  of  work  products, 
affected  organizations  must  have  a  shared  a  vision  of  the  working  relationship. 
Relationships  can  range  from  integrated  developer/supplier  product  teams,  to  prime- 
contractor/subcontractor,  to  vendors,  and  more.  A  successful  relationship  between  an 
organization  and  a  supplier  depends  on  the  capability  of  both  organizations,  and  on  a 
mutual  understanding  of  the  relationship  and  expectations. 


1 .  Process 
Maturity 


a.  In  your  opinion,  how  consistently  is  this  process  area  performed  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 

[  ]  This  process  area  is  consistently  performed 

( if  Never  performed,  go  to  the  next  process  area) 

b.  How  consistently  do  you  perform  the  following  base  practices  on  your  projects? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 


Consistency 

1.  [  ] 

2.  [  ] 

3.  [  ] 

4.  [  ] 

5.  [  ] 


Identify  needed  system  components  or  services  that  must  be  provided  by 
other/outside  organizations. 

Identify  suppliers  that  have  shown  expertise  in  the  identified  areas. 
Choose  suppliers  in  accordance  with  the  Analyze  Candidate  Solutions 
process  area  (PA  01). 

Provide  to  suppliers  the  needs,  expectations,  and  measures  of 
effectiveness  held  by  the  organization  for  the  system  components  or 
services  that  are  to  be  delivered. 

Maintain  timely  two-way  communication  with  suppliers. 
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4.  Key  Systems  Engineering  Process  Areas 


PA  18:  Coordinate  With  Suppliers,  Continued 


2 .  Process  a.  Does  your  project/organization  generally  Coordinate  With  Suppliers  to  a  sufficient 

Adequacy  level  to  ensure  a  cost  effective  system  which  satisfies  your  customers  requirements? 

(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don't  know) 

[  ]  Coordinate  with  Suppliers  sufficiently  performed 

b.  To  what  extent  do  the  following  factors  limit  the  effectiveness  of  the  process? 
(N=Never,  S=Seldom,  U=Usually,  0=0ften,  A=Always,  D=Don’t  know) 


Impact 


1. 

[ 

1 

Budget  limitations 

2. 

[ 

] 

Schedules  limitations 

3. 

[ 

] 

Policies/procedures 

4. 

[ 

] 

Methodologies/practices 

5. 

[ 

] 

Tool  availability 

6. 

[ 

] 

Tool  adequacy 

7. 

[ 

1 

Systems  engineering  personnel  availability 

8. 

[ 

] 

Engineering  personnel  availability 

9. 

[ 

] 

Training  and  personnel  experience 

10; 

[ 

] 

Other 

3 .  Applied  a.  What  methods  do  you  generally  use  to  Coordinate  With  Suppliers? 

Methodology  (Please  describe  and  if  required,  please  use  additional  paper  and  attach  to  the  end  of 
this  questionnaire) 


b.  What  tools  do  you  generally  use  to  Coordinate  With  Suppliers? 


continued  on  next  page 
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PA  18:  Coordinate  With  Suppliers,  Continued 


c.  What  internal,  industry  and  government  standards  do  you  follow? 


d.  What  metrics  does  your  organization  generally  apply  to  monitor  this  process? 


e.  Do  you  think  metrics  need  to  be  developed  for  this  process?  [Check  one] 

[  ]  Yes  -  Formal  metrics  to  assess  an  organization’s  performance  for  both 

internal  use  and  as  source  selection  criteria 
[  ]  Yes  -  Informal  metrics  to  assess  an  organization's  performance  for  internal 

use  only 

[  ]  No 

[  ]  Don’t  know 


4 .  Process  What  do  you  feel  are  the  top  two  items  which  would  improve  the  Coordinate  With 

Improvement  Suppliers  practice  in  your  organization? 


end  of  PA  18:  Coordinate  With  Suppliers 
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5.  Survey  Feedback 


5.  Survey  Feedback 


Respondent 

Information  Name:  _ Date: 

Title:  _ 

Phone:  _ 

Fax:  _ 

Project:  _ 

Organizational 

Information  Organization:  _ 

Address:  _ 


a.  Who  are  your  organization’s  customers?  [Check  all  that  apply] 

[  ]  Department  of  Defense 

[  ]  Civilian  Federal  Government 

[  ]  State  Government 

[  ]  Local  Government 

[  ]  Commercial 

[  ]  Industrial 

b.  How  much  time  did  you  spend  filling  out  this  questionnaire? 

[  ]  30  Min.  [  ]  2  Hours 

[  ]  60  Min.  [  ]  2.5  Hours 

[  ]  90  Min.  [  ]  (Other) _ 

c.  Which  Process  Areas  did  you  fill  out? 


[ 

] 

All 

[ 

] 

1 

[  ]  7 

[ 

] 

13 

[ 

] 

2 

[  ]  8 

[ 

] 

14 

[ 

] 

3 

[  ]  9 

[ 

] 

15 

[ 

] 

4 

[  ]  10 

[ 

] 

16 

[ 

] 

5 

[  ]  11 

[ 

] 

17 

[ 

] 

6 

[  ]  12 

[ 

] 

18 
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d  Where  the  Process  Areas  understandable 
[  ]  Yes  [  ]  No 

e.  If  not,  which  ones  and  why? 


f.  Where  the  instructions  clear? 

[  ]  Yes  [  ]  No 


g.  If  not,  why? 


h.  Was  the  format  easy  to  use? 

[  ]  Yes  [  ]  No 


i.  If  not,  why? 


continued  on  next  page 


Version  Ll.O 


76 


5.  Survey  Feedback 


Version  1.1.0 


77 


5,  Survey  Feedback 


This  page  intnetionally  left  blank. 


Version  1.1.0 


78 


